The present invention relates generally to graphical user interfaces, and more specifically to techniques for analyzing users' interactions with applications having such interfaces.
In 2011, Apple Inc. introduced “Smart Signs” into their retail stores. A Smart Sign is a tablet device used to replace the paper signs that would normally provide information regarding displayed products. In the specific implementation, the Smart Signs are tablet devices marketed by Apple under the iPad trademark. The application that runs on the Smart Sign tablets resembles in many ways a web browser in that a user (typically the customer in the retail context) interacts with the application by selecting various screen elements (“interactive objects” or “active objects”). The application that runs on the iPad to provide product information is sometimes referred to as the Smart Sign application.
The various regions of the display (often referred to as screen elements, screen objects, or simply objects) can be active, or they can be inactive. The term “object” often refers to a delineated region of the screen that sets it off from other regions. Objects are often displayed against a background, which users typically interpret as inactive, while it is not always self-evident from mere viewing whether objects defined by delineated regions set against the background are active or inactive.
When the user interacts with an active region, the application takes some action that is normally apparent to the user, e.g., changing something on the display. For example, the application can present a tabbed interface along an edge of the screen (i.e., a row of active regions styled to resemble notebook tabs), and when the user selects a tab, the application displays a different page with information suggested by the label on the selected tab. When the user attempts to interact with an inactive region, the application does nothing apparent, or in some applications, may make an unfriendly sound indicating that the application is not going to do anything that the user might have desired.
The iPad device incorporates a touchscreen, and the user interacts with screen objects by various touch interactions. The touch interactions can include, for example, a single tap, a double tap, a swipe, a long press, a multi-finger tap, or a multi-finger swipe. U.S. Patent Publication No. 2006/0026535 of Hotelling et al. for “Mode-Based Graphical User Interfaces for Touch Sensitive Input Devices” describes technology for interpreting different touch types. The entire disclosure of that patent publication is incorporated by reference.
A typical application in which embodiments of the present invention can be implemented, such as the Smart Sign application, has the capability of displaying various pages or screens, each of which can have one or more active objects or regions and one or more inactive objects or regions. Typically, a user interface (“UI”) event on an active object causes the application to change the attributes of some objects, to switch to displaying a different page, to display one or more elements (e.g., a dialog box or other menu item) superimposed on only a portion of the same page, or to move to a different portion of the same page. The different page can have a different set of active and inactive objects, although some might be the same as some on the other screen.